Image recognition method and apparatus, and computer-readable storage medium

ABSTRACT

Image recognition method and apparatus, and computer-readable storage medium are provided. The method includes: determining changed object according to change of states of at least one group of target objects in detection area; obtaining a set of image frames in side view consisting of image frames in side view and at least one image frame to be matched that are collected within a first preset time period before states of at least one group of target objects in detection area are changed; determining associated image frame from at least one image frame to be matched; obtaining image frame in side view corresponding to associated image frame from the set of image frame in side view; and determining, according to associated image frame and the image frame in the side view, target intervention object having the highest degree of association with changed object from at least one intervention object.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure is a U.S. continuation application ofInternational Application No. PCT/IB2020/053639, filed on Apr. 17, 2020,which claims priority to Singapore Patent Application No. 10201913955V,filed to the Singapore Patent Office on Dec. 31, 2019 and entitled“IMAGE RECOGNITION METHOD AND APPARATUS, AND COMPUTER-READABLE STORAGEMEDIUM”. The contents of International Application No. PCT/IB2020/053639and Singapore Patent Application No. 10201913955V are incorporatedherein by reference in their entireties.

TECHNICAL FIELD

The present disclosure relates to the artificial intelligence field, andin particular, relates to an image recognition method and apparatus, anda computer-readable storage medium.

BACKGROUND

In recent years, with the popularization and development of artificialintelligence technologies, image recognition performed by a computer orcamera is applied to more and more scenarios. For example, in currentvideo monitor and image recognition systems, a camera is used torecognize an association relationship between a target object andsurrounding multiple intervention objects, such as the saving andfetching relationships between a recognition item and a surroundingperson.

SUMMARY

The technical solution of the present disclosure is realized as below:

The embodiments of the present disclosure provide an image recognitionmethod, including:

determining a changed object according to a change of states of at leastone group of target objects in a detection area;

obtaining at least one image frame to be matched and a set of imageframes in a side view consisting of image frames in the side view thatare collected within a first preset time period before the states of theat least one group of target objects in the detection area are changed,wherein the at least one image frame to be matched is obtained bycollecting at least one image of the detection area by an imagecollection device located above the detection area, and the image framesin the side view are obtained by collecting images of the detection areaby an image collection device located on a side of the detection area;

determining an associated image frame from the at least one image frameto be matched, wherein the associated image frame comprises anintervention part having a highest degree of association with thechanged object;

obtaining an image frame in the side view corresponding to theassociated image frame from the set of image frame in the side view,wherein the image frame in the side view corresponding to the associatedimage frame comprises the intervention part having the highest degree ofassociation with the changed object, and at least one interventionobject; and

determining, according to the associated image frame and the image framein the side view corresponding to the associated image frame, a targetintervention object having the highest degree of association with thechanged object from the at least one intervention object.

The embodiments of the present disclosure provide an image recognitionapparatus, including:

a first determining unit, configured to determine a changed objectaccording to a change of states of at least one group of target objectsin a detection area;

a first obtaining unit, configured to obtain at least one image frame tobe matched and a set of image frames in a side view consisting of imageframes in the side view that are collected within a first preset timeperiod before the states of the at least one group of target objects inthe detection area are changed, wherein the at least one image frame tobe matched is obtained by collecting at least one image of the detectionarea by an image collection device located above the detection area, andthe image frames in the side view are obtained by collecting images ofthe detection area by an image collection device located on a side ofthe detection area;

a second determining unit, configured to determine an associated imageframe from the at least one image frame to be matched, wherein theassociated image frame comprises an intervention part having a highestdegree of association with the changed object;

a second obtaining unit, configured to obtain an image frame the sideview corresponding to the associated image frame from the set of imageframes in the side view, wherein the image frame in the side viewcorresponding to the associated image frame comprises the interventionpart having the highest degree of association with the changed object,and at least one intervention object; and

a third determining unit, configured to determine, according to theassociated image frame and the image frame in the side viewcorresponding to the associated image frame, a target interventionobject having the highest degree of association with the changed objectfrom the at least one intervention object.

The embodiments of the present disclosure provide an image recognitionapparatus, where the image recognition apparatus includes a processor, amemory, and a communication bus; the memory communicates with theprocessor by means of the communication bus; the memory stores one ormore programs that may be executed by the processor; and if the one ormore programs are executed, the processor executes the aforementionedimage recognition method.

The embodiments of the present disclosure provide a computer-readablestorage medium, where the computer-readable storage medium stores theone or more programs, and the one or more programs may be executed byone or more processors to realize the aforementioned any one imagerecognition method.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an image recognition scene provided bythe embodiments of the present disclosure.

FIG. 2 is an optional flow chart I of the image recognition methodprovided by the embodiments of the present disclosure.

FIG. 3 is an optional flow chart II of the image recognition methodprovided by the embodiments of the present disclosure.

FIG. 4 is an optional flow chart III of the image recognition methodprovided by the embodiments of the present disclosure.

FIG. 5 is an optional flow chart IV of the image recognition methodprovided by the embodiments of the present disclosure.

FIG. 6 is an optional flow chart V of the image recognition methodprovided by the embodiments of the present disclosure.

FIG. 7 is a schematic structural diagram I of the image recognitionapparatus provided by the embodiments of the present disclosure.

FIG. 8 is a schematic structural diagram II of the image recognitionapparatus provided by the embodiments of the present disclosure.

DETAILED DESCRIPTIONS

For current video monitor and image recognition systems, when anexisting camera recognizes an association relationship between a targetobject and surrounding multiple intervention objects, such as the savingand fetching relationships between a recognition item and a surroundingperson, the missing of the key information of a collected image causesthe problems, such as it is impossible to correctly associate to theintervention object that makes the target object change when the targetobject is changed, the collected information of the intervention objectis incomplete, and it is impossible to match a specific targetintervention object according to the information of a part ofintervention objects, and finally causes the problem that it isimpossible to correctly recognize the target intervention object havingthe highest degree of association with the target object.

For solving the aforementioned technical problems, embodiments of thepresent disclosure desire to provide an image recognition method andapparatus, and a computer-readable storage medium, and are capable ofimproving the accuracy of recognizing a target intervention object. Themethod includes: determining the changed object according to the changeof states of the at least one group of target objects in the detectionarea; obtaining the set of image frames in the side view consisting ofthe image frames in the side view and the at least one image frame to bematched that are collected within the first preset time period beforethe states of the at least one group of target objects in the detectionarea are changed, where the at least one image frame to be matched isobtained by collecting at least on image of the detection area by theimage collection device located above the detection area, and the imageframes in the side view are obtained by collecting images of thedetection area by an image collection device located on a side of thedetection area; determining the associated image frame from the at leastone image frame to be matched, where the associated image frame includesthe intervention part having the highest degree of association with thechanged object; obtaining an image frame in the side view correspondingto the associated image frame from the set of image frame in the sideview, where the image frame in the side view corresponding to theassociated image frame includes the intervention part having the highestdegree of association with the changed object, and the at least oneintervention object; and determining, on the basis of the associatedimage frame and the image frame in the side view corresponding to theassociated image frame, the target intervention object having thehighest degree of association with the changed object from the at leastone intervention object.

The solution is realized by using the aforementioned method. Theintervention part having the highest degree of association with thechanged object may be obtained from a bird's eye view. Because theposition information from the bird's eye view is proportional to actualposition information, a positional relationship between the changedobject and the intervention part obtained from the bird's eye view ismore accurate than that from a side view angle. Furthermore, theassociated image frame is combined with the corresponding side viewimage frame, thereby realizing the determining of the changed object andthe intervention part having the highest degree of association with thechanged object (the determining based on the associated image frame),further realizing the determining of the intervention part having thehighest degree of association with the changed object and the targetintervention object (the determining based on the corresponding sideview image frame), and thereby determining the target interventionobject having the highest degree of association with the changed object,and improving the accuracy of associating the change of the targetobject with the intervention object.

The technical solutions in the embodiments of the present disclosure areclearly and completely described below with reference to theaccompanying drawings in the embodiments of the present disclosure.

The embodiments of the present disclosure provide the image recognitionscene. As shown in FIG. 1, FIG. 1 is a schematic diagram of the imagerecognition scene provided by the embodiments of the present disclosure,including the image collection device 100, which is located above thedetection area 500, that generally performs image collection on thedetection area at a vertical angle in actual application; and the imagecollection device 200_1 and the image collection device 200_2, which arelocated at the sides of the detection area 500, that perform imagecollection on the detection area at a parallel angle in actualapplication, where the image collection device 100, the image collectiondevice 200_1, and the image collection device 200_2 continuously detectthe detection area 500 according to respective orientations and angles.At least one group of target objects 300_1 to 300_n are placed in thedetection area 500, where each group of target objects 300_1 to 300_nare formed by stacking at least one sub object. At least oneintervention object 400_1 to 400_n is included around the detection area500, where the intervention objects 400_1 to 400 are located within thecollection range of the image collection device 100, the imagecollection device 200_1, and the image collection device 200_2. In theimage recognition scene provided by the embodiments of the presentdisclosure, the image collection device may be a moving camera or astill camera, the intervention object may be a person, and the targetobject may be a stackable item. When a certain person among the persons400_1 to 400_n fetches or places the item in the detection area 500, thecamera 100 may capture the image that the handle of the person reachesinto the upper part of the detection area 500 from a vertical viewangle, and the camera 200_1 and the camera 200_2 may capture the imagesof the persons 400_1 to 400_n at different view angles at correspondingmoments.

In the embodiments of the present disclosure, the image collectiondevice 100 generally is provided above the detection area 500, such asbeing directly above or in the vicinity directly above the central pointof the detection area, and the collection range at least covers thewhole detection area; the image collection devices 200_1 and 200_2 arelocated at the sides of the detection area, are respectively provided attwo opposite sides of the detection area, and have the setting heightsthat are flush with the target object in the detection area, and thecollection range covers the whole detection area and the interventionobjects around the detection area.

In some embodiments, if the detection area is a square area on adesktop, the image collection device 100 may be provided directly abovethe central point of the square area, the setting height thereof may beadjusted according to the view angle of the specific image collectiondevice, and it is guaranteed that the collection range may cover thesquare area of the whole detection area; the image collection devices200_1 and 200_2 are respectively provided at two opposite sides of thedetection area, the setting heights thereof are flush with the targetobjects 300_1 to 300_n of the detection area, the distance between theimage collection device and the detection area may be adjusted accordingto the view angle of the specific image collection device, and it isguaranteed that the collection range may cover the whole detection areaand the intervention objects around the detection area.

It should be noted that in actual use, apart from the image collectiondevices 200_1 and 200_2, more image collection devices located at theside of the detection area may also be set according to requirements.The embodiments of the present disclosure do not define this.

FIG. 2 is an optional flow chart of the image recognition methodprovided by the embodiments of the present disclosure, and descriptionis made with reference to the operation shown in FIG. 2.

At S101, the changed object is determined according to the change ofstates of the at least one group of target objects in the detectionarea.

The image recognition method provided by the embodiments of the presentdisclosure is suitable for recognizing the target object or person thatmakes a monitored object change in multiple objects or persons.Illustratively, the image recognition method provided by the embodimentsof the present disclosure is suitable for the scenes, such as automaticbookkeeping and retail systems, and intelligent item monitoring.

In the embodiments of the present disclosure, each group of targetobjects in the at least one group of target objects may be formed bystacking at least one sub object.

In some embodiments of the present disclosure, at least one group oftarget objects may be stacked some piles of books; each pile of books isa group of target objects and may also be stacked coins; and each pileof coins are a group of target objects.

In the embodiments of the present disclosure, the image collectiondevice may be provided above the detection area, and is configured tomonitor the sub object included in the at least one group of targetobjects, and determining that the states of the at least one group oftarget objects are changed when the sub object included in the at leastone group of target objects is changed, or it is monitored that a newgroup of target object occur in the detection area, or any one group ofprevious target objects disappear in the detection area.

In the embodiments of the present disclosure, the image collectiondevice may also be provided at an angle parallel to the side of thedetection area, and is configured to continuously monitor the sub objectincluded in the at least one group of target objects, and determiningthat the states of the at least one group of target objects are changedwhen it is monitored that the number or the appearance of the subobjects included in any one group of target objects is changed.

It should be noted that in the embodiments of the present disclosure,the problem that the target objects shelter each other exists when it ismonitored is performed from the side view angle, and therefore, in orderto ensure the accuracy of monitoring, at least two image collectiondevices may be provided at the sides of the detection area and monitorthe number or the appearance of the at least one existing group oftarget objects in the detection area.

In the embodiments of the present disclosure, all detection areas may beplaced in the detection frame of the image collection device; if thenumber of at least one group of target objects is not changed in thedetection frame, the image statistical characteristics of the detectionarea may be represented as a smoothing curve on continuous collectiontime points; if the number of at least one group of target objects ischanged in the detection frame, for example, the total number of groupsincreases or decreases, the curve corresponding to the imagecharacteristic of the at least one group of target objects is changedabruptly, and it is indicated that the image content of the at least onegroup of target objects is changed, and thus, it is known whether the atleast one group of target objects in the detection area are changed bymonitoring the image characteristic curve of the at least one group oftarget objects.

In the embodiments of the present disclosure, in order to ensure theaccuracy of detection, it can be determined that the states of the atleast one group of target objects are changed after the number of atleast one group of target objects that are changed stably exceeds thepreset number of frames.

In some embodiments, after it is detected that the number of newly addedat least one group of target objects stably exceeds 5 frames, or thenumber of reduced at least one group of target objects stably exceeds 20frames, it is determined that the number of at least one group of targetobjects is changed.

In some embodiments, three piles of stacked books are taken as at leastone group of target objects, where each pile of books corresponds to agroup of target objects; one pile of books include 20 books; a side viewcamera is taken as the image collection device located at the angleparallel to the side of the detection area, and separately performscontinuous image collection on the detection area from the left andright side angles; if 5 books are taken away from the pile of booksincluding 20 books, the side view camera monitors that the number ofbooks in one pile of books including 20 books among three piles ofstacked books decreases; and if the reduced 5 books disappear more than20 frames from the detection area, it is determined that the number ofat least one sub object in the at least one group of target objects ischanged.

In the embodiments of the present invention, the at least one sub objecthaving the change of states in the at least one group of target objectsis determined as the changed object.

In some embodiments, a camera with aerial view is the image collectiondevice located above the detection area; if the camera with aerial viewdetects that a group of target objects are added in a new position ofthe detection area, a group of target objects is added and taken as thechanged objects; and if the camera with aerial view detects that onegroup of previous target objects in the detection area disappear, thedisappeared target objects are taken as the changed objects.

In some embodiments, the side view camera is the image collection devicelocated at the side of the detection area; if the side view cameradetects that the number of sub objects in one existing group of targetobjects increases by 20, the increased 20 sub objects are taken as thechanged objects; and if the side view camera detects that the number ofsub objects in one existing group of target objects decreases by 20, thedecreased 20 sub objects are taken as the changed objects.

At S102, the set of image frame in the side view consisting of the imageframes in the side view and the at least one image frame to be matchedthat are collected within the first preset time period before the statesof the at least one group of target objects in the detection area arechanged are obtained, where the at least one image frame to be matchedis obtained by collecting at least one image of the detection area by animage collection device located above the detection area, and the imageframes in the side view are obtained by collecting images of thedetection area by an image collection device located on a side of thedetection area.

In the embodiments of the present disclosure, if the states of the atleast one group of target objects in the detection area are changed, theset of image frame in the side view and the at least one image frame tobe matched that are collected within the first preset time period beforethe states of the at least one group of target objects are changed areobtained by means of the image collection device.

In the embodiments of the present disclosure, the image collectiondevice is configured to perform continuous image collection on thedetection area and the intervention objects around the detection. Theimage collection device in the embodiments of the present disclosure mayalso be a moving camera, may also be a still camera, or is other deviceshaving the image collection function. The embodiments of the presentdisclosure do not define this.

In the embodiments of the present disclosure, the image collectiondevice is provided above the detection area and configured tocontinuously monitor the image of the at least one group of targetobjects from the bird's eye view; and the image collection device isprovided at the side of the detection area and configured tocontinuously monitor the image of the at least one group of targetobjects from the side view angle. If the states of the at least onegroup of target objects are changed, the image frames that are collectedwithin the first preset time period before a state change time point areobtained by means of the image collection device located above thedetection area and taken as at least one image frame to be matched;meanwhile, the image frame that is synchronously collected with the atleast one image frame to be matched is obtained by means of the imagecollection device located at the side of the detection area and taken asthe set of image frame in the side view.

In some embodiments, the camera with aerial view provided above thedetection area is taken as the image collection device; the side viewcamera provided at the side of the detection area is taken as the imagecollection device; the number of at least one group of target objects ischanged at time point B, and the first preset time period is 2 seconds;the image frame that is collected by the camera with aerial view fromB-2 time point to B time point is obtained and taken as the at least oneimage frame to be matched; and the image frame that is collected by theside view camera from B-2 time point to B time point is obtained andtaken as the set of image frame in the side view.

It can be understood that in the embodiments of the present disclosure,if the number of at least one group of target objects in the detectionarea is changed, the images of at least one group of target objects thatare intervened to change can be captured from the bird's eye view, andthus, the problem that the target objects shelter each other from theside view angle is avoided; meanwhile, the whole side image of at leastone intervention object near the target object can be captured from theside view angle, and thus, the problem of failing to capture the wholeimage of the intervention object from the bird's eye view is avoided.

At S103, the associated image frame is determined from the at least oneimage frame to be matched, where the associated image frame includes theintervention part having the highest degree of association with thechanged object.

In the embodiments of the present disclosure, after the changed objectis determined, the associated image frame having the highest degree ofassociation with the changed object is determined from the at least oneimage frame to be matched.

In the embodiments of the present disclosure, based on FIG. 2,determining the associated image frame having the highest degree ofassociation with the changed object from the at least one image frame tobe matched may be as shown in FIG. 3, including the following operationsS1031-S1034.

At S1031, at least one intervention part is recognized from each of atleast one image frame to be matched, where the at least one interventionpart is a part of the at least one intervention object which appears inat least one image frame to be matched.

In the embodiments of the present disclosure, after the at least oneimage frame to be matched is obtained, the recognition of theintervention part is performed on each image frame to be matched, andthe intervention part in each image frame to be matched is recognized,and thus, at least one intervention part is obtained.

In the embodiments of the present disclosure, an image recognitionalgorithm may be configured to recognize at least one intervention partfrom each image frame to be matched, and other methods may also be used.The embodiments of the present disclosure do not define this.

In the embodiments of the present disclosure, one intervention part is apart where the state of the target object is changed because oneintervention object intervenes the target object, and is a part thatappears in the at least one image frame to be matched.

In the embodiments of the present disclosure, when a part of theintervention objects enter the detection area of the image collectiondevice, whether the intervention objects contact the changed object ornot, a part of images of the intervention objects can be captured in theimage frame to be matched, and the part of the captured image of atleast one intervention object is taken as at least one interventionpart.

In some embodiments, the camera with aerial view serves as the imagecollection device located above the detection area. When theintervention object is a person, the image that a human hand stretchesinto the detection area can be captured by means of the camera withaerial view, and at least one human hand image that is captured in theimage frame to be matched is taken as at least one intervention part.

At S1032, the changed object is combined with the recognizedintervention part in the one-to-one manner to obtain at least onecombination of changed object and intervention part.

In the embodiments of the present disclosure, after at least oneintervention part is recognized, the changed object is combined with therecognized intervention part in the one-to-one manner to obtain at leastone combination of changed object and intervention part.

It should be noted that in the embodiments of the present disclosure,because it is needed to recognize the intervention part included in theimage frame to be matched from each image frame to be matched, at leastone intervention part may include several repeated intervention parts,and when the changed object is combined with at least one interventionpart in the one-to-one manner, it is needed to guarantee that thechanged object is combined with each intervention part only once.

At S1033, the combination having the highest degree of association isdetermined from at least one combination of changed object andintervention part, where the intervention part in the combination havingthe highest degree of association is the target intervention part, thatis to say, the target intervention part is the intervention part havingthe highest degree of association with the changed object in theintervention parts.

In the embodiments of the present disclosure, the determining thecombination having the highest degree of association from thecombination of at least one group of changed objects and an interventionobject part can be realized by means of the following operations S201 toS202.

At S201, the distances between the changed objects and the interventionparts in each of the at least one combination of changed object andintervention part are obtained.

In the embodiments of the present disclosure, the distances between thechanged objects and the intervention parts in each of the at least onecombination of changed object and intervention part can be obtained bymeans of the image collection device located above the detection area.

In the embodiments of the present disclosure, in the side view angle,the distance between the intervention part and the changed object is notproportional to an actual distance, and therefore, the actual distancecannot be calculated according to the distance between the interventionpart and the changed object obtained from the side view angle. Thedistances between the changed objects and the intervention parts in thecombination of each changed object and the intervention part is obtainedfrom the bird's eye view by means of the image collection device locatedabove the detection area to obtain the distance between at least onechanged object and the intervention part.

At S202, a minimum distance is determined from the distance.

In the embodiments of the present disclosure, a minimum value is foundfrom the distance between at least one changed object and theintervention part, and taken as the minimum distance.

In some embodiments of the present disclosure, the method fordetermining the minimum distance from the distance between at least onechanged object and the intervention part further can include thefollowing operations S2021 to S2022.

At S2021, inverse ratio calculation is performed on the distance toobtain the calculation result of each distance in the distance betweenat least one changed object and the intervention part.

In the embodiments of the present disclosure, any formula that isnegatively correlated with the distance can be used for calculation inthe method for performing inverse ratio calculation on the distance toobtain a corresponding calculation result.

In the embodiments of the present disclosure, functions that arenegatively correlated with the distance all can be used for calculation,and the embodiments of the present disclosure do not define this.

In some embodiments, the distances between the changed objects and theintervention parts in each combination is d, and the calculation resultof a distance inverse ratio calculation can be obtained by calculating

$\frac{1}{d}$

or e^(−d).

In the embodiments of the present disclosure, the method for performinginverse ratio calculation on the distance may also be other method, andthe embodiments of the present disclosure do not define this.

At S2022, a maximum calculation result is determined from thecalculation result, and the distance corresponding to the maximumcalculation result is determined as the minimum distance.

In the embodiments of the present disclosure, after the result of thedistance inverse ratio calculation is obtained, the maximum calculationresult is determined from the calculation result, and the distancecorresponding to the maximum calculation result is determined as theminimum distance.

It can be understood that the intervention part corresponding to themaximum calculation result of the distance inverse ratio calculation isthe intervention part closest to the changed object.

At S203, the combination of the changed object and the intervention partcorresponding to the minimum distance is taken as the combination havingthe highest degree of association.

In the embodiments of the present disclosure, the intervention partcorresponding to the minimum distance is the intervention part closestto the changed object in at least one intervention part, and theprobability that the intervention part is associated with the changedobject is maximum Therefore, the combination of the changed object andthe intervention part corresponding to the minimum distance is taken asthe combination having the highest degree of association.

In some embodiments, when one pile of books are added to the stackedbooks in the detection area, at least one group of image frames to bematched are obtained by means of the camera with aerial view, and atleast one human hand image of entering the detection is captured in atleast one group of image frames to be matched. The added books arecombined with at least one human hand image in a one-to-one manner. Thedistances between the added books and the human hand image arecalculated, and the combination of the added book corresponding to theminimum distance and the human hand image taken as the combinationhaving the highest degree of association.

It can be understood that in the embodiments of the present disclosure,the intervention part in the combination having the highest degree ofassociation is the intervention part that is most likely to contact thechanged object. Because the distance obtained from the bird's eye viewis proportional to the actual distance, the present disclosure solvesthe problem that the distance between the intervention part and thechanged object obtained from the side view angle is disproportionate,and the problem that it is not accurate to associated the interventionpart and the changed object from the side view angle, thereby improvingthe accuracy of associating the changed object and the interventionpart, and further improving the accuracy of associating the changedobject and the target intervention object.

In some embodiments of the present disclosure, the determining thecombination having the highest degree of association from at least onecombination of changed object and intervention part further can includethe following operations S301 to S302.

At S301, an actual motion trajectory of each of the recognizedintervention parts is obtained.

In the embodiments of the present disclosure, the obtaining the actualmotion trajectory of each of the recognized intervention parts caninclude the following operations S3011 to S3014.

At S3011, the trigger time point when the state of the target object ischanged is obtained.

In the embodiments of the present disclosure, the time point when thestate of the target object is changed is recorded and obtained as thetrigger time point.

At S3012, at least one motion trajectory image frame that is collectedwithin a second preset time period before the trigger time point isobtained, and the at least one motion trajectory image frame is obtainedby the image collection device located above the detection area.

In the embodiments of the present disclosure, after the trigger timepoint, at least one image frame that is collected within the secondpreset time period before the trigger time point is obtained and takenas at least one motion trajectory image frame.

In some embodiments, the intervention part is the human hand image; thesecond preset time period is 5 seconds; 10 image frames collected by thecamera with aerial view within 5 seconds are obtained; each image frameincludes at least one human hand image located at different positions;and the 10 image frames are taken as at least one motion trajectoryimage frame.

It should be noted that in the embodiments of the present invention, inorder to make the motion trajectory of the intervention part be moreaccurate, the second preset time period is generally set to be the timeperiod longer than the first preset time period. Therefore, at least onemotion trajectory image frame that is collected within the second presettime period before the trigger time point generally includes at leastone image frame to be matched that is collected within the first presettime period.

At S3013, the corresponding position information of each recognizedintervention part in at least one motion trajectory image frame isobtained.

In the embodiments of the present disclosure, at least one motiontrajectory image frame includes the images of each intervention part atdifferent positions, and for the same intervention part, the respectivecorresponding position information of the intervention part in eachmotion trajectory image frame is obtained from at least one motiontrajectory image frame.

In the embodiments of the present disclosure, for each interventionpart, the respective corresponding position information of eachintervention part in each motion trajectory image frame is determined.

In some embodiments, when the intervention part is the human hand image,the position of the human hand image in each image frame of 10 frames ofmotion trajectory image frames is obtained and taken as the positioninformation corresponding to the image of the human hand.

At S3014, according to a time sequence of at least one motion trajectoryimage frame, vectorization connection is performed on the positioninformation to obtain the actual motion trajectory of each recognizedintervention part.

In the embodiments of the present disclosure, for the same interventionpart, after the position information corresponding to the interventionpart is determined, the position information corresponding to theintervention part is sequentially connected to be a line in the sequencefrom the starting point to the ending point of the time sequenceaccording to a time sequence of at least one motion trajectory imageframe, so that a vector trajectory corresponding to the interventionpart is obtained and taken as the actual motion trajectory of theintervention part.

In the embodiments of the present disclosure, for each recognizedintervention part, the actual motion trajectory of each recognizedintervention part is determined.

In some embodiments, after the position information corresponding to onehuman hand image is obtained, according to different collection time of10 frames of motion trajectory image frames, connection is sequentiallyperformed starting from the position corresponding to the human handimage in a first motion trajectory image frame to the positioncorresponding to the human hand image in next motion trajectory imageframe, and finally the actual motion trajectory corresponding to thehuman hand image is obtained.

At S302, in at least one combination of changed object and interventionpart, a simulated motion trajectory from the intervention part to thechanged object in each combination is obtained, where the simulatedmotion trajectory from the intervention part to the changed object inone combination is a trajectory connecting the intervention part and thechanged object in the combination.

In the embodiments of the present disclosure, for the combination of onechanged object and the intervention part, the intervention part in thecombination is connected to the changed object, and the obtained vectortrajectory is taken as the simulated motion trajectory from theintervention part to the changed object in the combination.

At S303, similarity between each actual motion trajectory and eachsimulated motion trajectory is compared.

In the embodiments of the present disclosure, for the combination of onechanged object and the intervention part, the similarity between theactual motion trajectory of the intervention part included in thecombination and the simulated motion trajectory corresponding to thecombination is compared.

In the embodiments of the present disclosure, the method for comparingthe similarity between two vector trajectories is the prior art, and theembodiments of the present disclosure do not describe it again.

At S304, the combination of the changed object and the intervention partcorresponding to the simulated motion trajectory having the highestsimilarity with the actual motion trajectory is determined to be thecombination having the highest degree of association

In the embodiments of the present disclosure, the actual motiontrajectory having the highest similarity with the simulated motiontrajectory is most likely to be the motion trajectory moving towards thechanged object and contacting the changed object. Therefore, thecombination of the changed object and the intervention partcorresponding to the simulated motion trajectory having the highestsimilarity with one actual motion trajectory is determined to be thecombination having the highest degree of association.

It should be noted that operations S201 to S203 and S301 to S303 relateto two optional methods for determining the combination having thehighest degree of association from at least one combination of changedobject and intervention part, and one or two methods can be selectedaccording to an actual situation when the methods are specificallyapplied. The embodiments of the present disclosure do not define this.

At S1034, the image frame to be matched in which the intervention partin the combination having the highest degree of association located istaken as the associated image frame. That is to say, the associatedimage frame includes the intervention part having the highest degree ofassociation with the changed object, i.e., including the targetintervention part.

In the embodiments of the present disclosure, after the combinationhaving the highest degree of association, the image frame to be matchedin which the intervention part in the combination having the highestdegree of association is located is taken as the associated image frame.That is to say, the associated image frame includes the information ofthe intervention part having the highest degree of association with thechanged object.

After operation S103, as shown in FIG. 2 and FIG. 3, the imagerecognition method provided by the embodiments of the present disclosurefurther includes the following operations.

At S104, the image frame in the side view corresponding to theassociated image frame is obtained from the set of image frame in theside view, where the image frame in the side view corresponding to theassociated image frame includes the intervention part having the highestdegree of association with the changed object, and at least oneintervention object.

In the embodiments of the present disclosure, after the associated imageis obtained, the image frame in the side view corresponding to theassociated image frame that is synchronously collected with theassociated image frame is obtained from the set of image frame in theside view.

In the embodiments of the present disclosure, the image collectiondevice located above the detection area and the image collection devicelocated on the side of the detection area synchronously performcollection on the detection area. Therefore, at the time point when theimage collection device located above the detection area collects theassociated image, the image collection device located on the side of thedetection area can also synchronously collect the image frame in theside view corresponding to the associated image frame of the same scene.

In the embodiments of the present disclosure, the whole side image of atleast one intervention object associated with the changed object can becaptured from the image frame in the side view corresponding to theassociated image frame.

In some embodiments, when detection is performed on the actions that oneperson takes and puts the book of the detection area, it is determined,by means of the camera with aerial view, that the number of books isvaried, then the human hand image having the highest degree ofassociation with the changed object in the detection area is entered,and the image frame in which the human hand image is located is taken asthe associated image frame. The image frame in the side viewcorresponding to the associated image frame that is synchronouslycollected with the associated image frame is obtained in the side viewcamera, where the image frame in the side view corresponding to theassociated image frame includes the side image of at least one person,including the side face image of at least one person, the image of sidebody of at least one person, and the side human hand image of at leastone person. In the side image of at least one person, the side faceimage, the image of side body, and the side human hand image of eachperson are associated with each other.

It can be understood that in the embodiments of the present disclosure,an intervention part image included in the associated image frame is theimage collected from the bird's eye view, because it is very difficultto collect the key identification information, such as human faceinformation, of the intervention object from the bird's eye view, it isimpossible to directly associate the changed object with theidentification information of the intervention object only according tothe intervention part image from the bird's eye view. Therefore, it isneeded to obtain a synchronous side image frame configured to associatethe intervention part image from the bird's eye view with the whole sideimage of the intervention object.

At S105, the target intervention object having the highest degree ofassociation with the changed object is determined from at least oneintervention object on the basis of the associated image frame and theimage frame in the side view corresponding to the associated imageframe.

In the embodiments of the present disclosure, after the imagerecognition device obtains the image frame in the side viewcorresponding to the associated image frame, the image recognitiondevice performs image fusion on the associated image frame and the imageframe in the side view corresponding to the associated image frame tofuse the image information of the intervention part and the interventionobject synchronously obtained from different angles together, therebydetermining the target intervention object having the highest degree ofassociation with the changed object from at least one interventionobject, and completing the image recognition process of associating thechanged object to the target intervention object.

In the embodiments of the present disclosure, based on FIG. 3, accordingto the associated image frame and the image frame in the side viewcorresponding to the associated image frame, the determining the targetintervention object having the highest degree of association with thechanged object from at least one intervention object may be as shown inFIG. 4, and includes the following operations S1051 to S1053.

At S1051, image fusion is performed on the associated image frame andthe image frame in the side view corresponding to the associated imageframe to obtain a side view image of the target intervention part, andthe side view image of the target intervention part is the image of thetarget intervention part in the image frame in the side viewcorresponding to the associated image frame.

In the embodiments of the present disclosure, because the associatedimage frame and the image frame in the side view corresponding to theassociated image frame are the image frames obtained by performingsynchronous collection on the detection area from different angles, theimage information of the same object obtained from the bird's eye viewand the side view angle are associated with each other, so that theimage at the side view angle corresponding to the target interventionpart at the bird's eye view in the image frame in the side viewcorresponding to the associated image frame is obtained and taken as theside image of the target intervention part.

In the embodiments of the present disclosure, the intervention part isthe human hand image; the intervention object is the person; the targetintervention part included in the associated image frame at the bird'seye view is the human hand image A1; and a side view person image frameincludes a side human hand image A1′, a side human hand image B1′, and aside human hand image Cl. After image fusion is performed on theassociated image frame and the image frame in the side viewcorresponding to the associated image frame, and image fusion isperformed on human hand image information obtained from the bird's eyeview and the human hand image information obtained from the side viewangle, the information that the side human hand image A1′ and the humanhand image A1 obtained from the bird's eye view are the same human handcan be obtained, and the side human hand image A1′ is taken as the sideimage of the target intervention part.

At S1052, the side view image of the target intervention objectassociated with the side view image of the target intervention part isobtained on the basis of the image frame in the side view correspondingto the associated image frame, where the image frame in the side viewcorresponding to the associated image frame includes the side view imageof at least one intervention object.

In the embodiments of the present disclosure, the image frame in theside view corresponding to the associated image frame includes the wholeside image of at least one intervention object, and the side view imageof the target intervention part is a part of the side image of oneintervention object, that is to say, the side view image of the targetintervention part is a part of the side view image of the correspondingintervention object, and is associated with other parts of the side viewimage of the intervention object. On the basis of the image frame in theside view corresponding to the associated image frame, the side viewimage of the intervention object associated with the side view image ofthe target intervention part is taken as the side view image of thetarget intervention object.

In some embodiments, the side view image of the target intervention partis the side human hand image A1′, and the image frame in the side viewcorresponding to the associated image frame includes a side image ofperson A′, a side image of person B′, and a side image of person C′. Theside image of person A′ further includes an image of side body A2′ and aimage of side face A3′. Because in the image frame in the side viewcorresponding to the associated image frame, the human hand image A1′ atthe side view angle and the image of side body A2′ are connected witheach other, A1′ can be associated to A2′; the image of side body A2′ isalso connected with the image of side face A3′, and then the human handimage A1′ at the side view angle is further associated to the image ofside face A3′, and the image of side face A3′ is taken as the side viewimage of the target intervention object.

It can be understood that after multi-angle image fusion is performed onthe associated image frame in which the target intervention part islocated and the image frame in the side view corresponding to theassociated image frame, an association relationship between the targetintervention part at the bird's eye view and the side view image of thetarget intervention object at the side view angle is obtained, and thusidentity recognition can be further performed on the target interventionobject according to the information included in the side view image ofthe target intervention object, so that the problem that theidentification information of the intervention object cannot becollected at the bird's eye view, and the problem that the interventionpart image at the bird's eye view cannot be directly associated with theidentification information of the intervention object are solved.

At S1053, according to the side view image of the target interventionobject and a pre-stored corresponding relationship between images ofintervention objects and identity information of the interventionobjects, the identity information of the target intervention object isdetermined.

In the embodiments of the present disclosure, for one interventionobject, the corresponding relationship between images of interventionobjects and identity information of the intervention objects ispre-stored, and the identity information of the intervention object canbe uniquely corresponded by means of the image of the interventionobject.

In some embodiments, the pre-stored corresponding relationship betweenimages of intervention objects and identity information of theintervention objects may be the corresponding relationship between theimage of frontal face of each person and the ID of the person;illustratively, the image of frontal face A corresponds to the person A,the image of frontal face B corresponds to the person C, and the imageof frontal face C corresponds to the person C. According to the sideview image of the target intervention object, i.e., the human face imageA3′ at the side view angle, the corresponding image of frontal face Acorresponding to A3′ is matched; then the human face image A3′ at theside view angle is further associated to the person A corresponding tothe image of frontal face A by means of the pre-stored correspondingrelationship between the image of frontal face of each person and the IDof the person, and the person A is taken as the target interventionobject.

It can be understood that in the embodiments of the present disclosure,the information of the target intervention part having higher accuracyand associated with the changed object can be obtained from the bird'seye view, and by means of image fusion, the information of the targetintervention part at the bird's eye view is combined with the completeinformation of the intervention object at the side view angle, so thatthe present disclosure realizes the association from the information ofthe target intervention part at the bird's eye view to the informationof the target intervention object at the side view angle, furtherassociating the information of the target intervention object with thepre-stored identification information of the intervention object, andimproving the accuracy of recognizing the target intervention object.

In the embodiments of the present disclosure, after S105, the followingoperations S106 to S107 are further included.

At S106, a side image of the changed object is obtained from the imageframe in the side view corresponding to the associated image frame.

At S107, an object value of the changed object is obtained according tothe side image of the changed object, where the object value is the sumof values of at least one sub changed object constituting the changedobject, the side image of the changed object includes a side image ofeach of at least one sub changed object, and the side image of each ofat least one sub changed object represents different values.

In the embodiments of the present disclosure, at least one target objectis formed by stacking at least sub object and the changed object is apart of at least one group of target objects, and therefore, the imageof the changed object at the side view angle can be obtained by means ofthe image collection device at the side of the detection area, andaccording to the images of the changed object at different side viewangles, the object value of the changed object is obtained.

In the embodiments of the present disclosure, the changed objectincludes at least one sub changed object, where the image of the subchanged object at the side view angle may include different colors,figures, or recognizable patterns or characters; different colors,figures, or recognizable patterns or characters may represent differentvalue information.

In the embodiments of the present disclosure, the object value of thechanged object is the sum of the values represented by the image of eachsub changed object at the side view angle constituting the changedobject.

At S108, the target intervention object is associated with the objectvalue, so that the information of a corresponding object value that isobtained or payed by the target intervention object is obtained.

In some embodiments, the value of each red book is RMB 10 yuan and thevalue of each green book is RMB 20 yuan; by means of the camera at theside view angle, it is detected that the height of a certain pile ofstacked books is reduced and the number of books is reduced; by means ofthe side view camera, the side image of a reduced book is obtained; bymeans of the side colors of the books, it is found that 10 red books and5 green books are reduced, and it can be obtained that the object valueof the reduced books is RMB 200 yuan; according to S101 to S105, it isobtained that the target intervention object is the person A; and thetarget intervention object is associated with the object value, so thatthe information that the person A takes away the books having the valueof RMB 200 yuan.

It can be understood that the object values of at least one group ofchanged objects are obtained by means of the camera at the side viewangle and are associated with a target person; and the specificinformation that the target person operates at least one group ofchanged objects is further complemented.

In some embodiments of the present disclosure, an implementation methodof recognizing the target intervention object associated with thechanged object, and associating the target intervention object with theobject value may be as shown in FIG. 5, and includes the followingoperations S601 to S616.

At S601, when it is monitored by means of the image collection devicethat a new group of target objects appear in the detection area, orprevious any of the detection area disappear, the changed object isdetermined from at least one group of target objects.

At S602, at least one image frame to be matched that is collected withinthe first preset time period before the current time point and the setof image frame in the side view are obtained.

At S603, at least one intervention part is recognized from each imageframe to be matched.

At S604, the changed object is combined with the recognized interventionpart in the one-to-one manner to obtain at least one combination ofchanged object and intervention part.

At S605, the distances between the changed objects and the interventionparts in each of the at least one combination of changed object andintervention part are obtained.

At S606, inverse ratio calculation is performed on the distance toobtain the calculation results of each of the distances.

At S607, the maximum calculation result is determined from thecalculation result, and the distance corresponding to the maximumcalculation result is determined as the minimum distance.

At S608, the combination of the changed object and the intervention partcorresponding to the minimum distance is taken as the combination havingthe highest degree of association.

At S609, the image frame to be matched in which the intervention part inthe combination having the highest degree of association is located istaken as the associated image frame.

At S610, the intervention part in the combination having the highestdegree of association is determined as the target intervention part.

At S611, image fusion is performed on the associated image frame and theimage frame in the side view corresponding to the associated image frameto obtain a side view image of the target intervention part, and theside view image of the target intervention part is the image of thetarget intervention part in the image frame in the side viewcorresponding to the associated image frame.

At S612, the side view image of the target intervention objectassociated with the side view image of the target intervention part isobtained on the basis of the image frame in the side view correspondingto the associated image frame, where the image frame in the side viewcorresponding to the associated image frame includes the side view imageof at least one intervention object.

At S613, according to the side view image of the target interventionobject and a pre-stored corresponding relationship between images ofintervention objects and identity information of the interventionobjects, the target intervention object is determined.

At S614, the side image (i.e., the image at the side view angle) of thechanged object is obtained from the image frame in the side viewcorresponding to the associated image frame.

At S615, the object value of the changed object is obtained according tothe side image of the changed object.

At S616, the target intervention object is associated with the objectvalue to complete the process of recognizing the target interventionobject associated with the changed object, and associating the targetintervention object with the object value.

It should be noted that the image recognition method provided by theembodiments of the present disclosure relates to continuously monitoringat least one target object in the detection area after recognizing thetarget intervention object associated with the changed object, and whenthe amount of at least one target object is changed again, continuouslyrecognizing the target intervention object in the same processingmethod.

The embodiments of the present disclosure also provide an imagerecognition method that is suitable for the scene of intelligentlymonitoring to put or withdraw a chip. On the basis of the schematicdiagram of the image recognition scene in FIG. 1, at least one group oftarget objects in the embodiments of the present disclosure may be atleast one group of chips, and correspond to the target objects 300_1 to300_n in FIG. 1; at least one intervention object may be at least oneplayer participating in a game, certainly may also be a croupier, andcorresponds to the intervention objects 400_1 to 400_n in FIG. 1; theimage collection device located above the detection area may be thecamera with aerial view, and corresponds to the image collection device100 in FIG. 1; and the image collection device located on the side ofthe detection area may be the side view camera, and corresponds to theimage collection devices 200_1 and 200_2 in FIG. 1. The imagerecognition method provided by the embodiments of the present disclosuremay be as shown in FIG. 6, and includes the following operations S701 toS721.

At S701, when it is detected that one group of chips are stacked on anexisting group of chips in the detection area again, it is determined bymeans of the side view camera that the number of at least one group ofchips in the detection area is changed.

In the embodiments of the present invention, at least one group oftarget objects are at least one group of chips.

In the embodiments of the present invention, when one group of chips arestacked on an existing group of chips in the detection area again, itcan be found by means of the side view camera that the number of chipsincluded in the existing group of chips in the detection area isincreased, and the state of at least one group of chips in the detectionarea is determined to be changed.

At S702, a newly added chip is determined from at least one group ofchips in the detection area.

In the embodiments of the present invention, the changed object is thenewly added chip.

At S703, 3 image frames collected within 2 seconds before the currenttime point are obtained by means of the camera with aerial view, and aretaken as at least one image frame to be matched; and the image framecollected by the side view camera within the corresponding time periodis obtained and taken as the set of image frame in the side view.

In the embodiments of the present disclosure, the first time period is 2seconds.

At S704, at least one human hand image is recognized from each imageframe to be matched.

In the embodiments of the present invention, at least one interventionpart is at least one human hand image.

It can be understood that in the embodiments of the present disclosure,when one group of chips are stacked on the existing group of chips inthe detection area again, the human hand necessarily stretches into thedetection area, at least one image frame to be matched before the timepoint when the number of at least one group of chips is changed iscaptured by means of the camera with aerial view, and at least one humanhand image may be recognized from at least one image frame to bematched, where at least one human hand image includes the human handimage where the chips are stacked, and may also include other human handimages. Therefore, it is needed to determine, from at least one humanhand image, a human hand where the chips are stacked.

At S705, newly added chip images at the camera with aerial view arecombined with at least one human hand image in the one-to-one manner toobtain the combination of at least one group of newly added chip imagesand the human hand image.

In the embodiments of the present disclosure, the newly added chipimages are combined with at least one human hand image in the one-to-onemanner, and it is guaranteed that the newly added chip images arecombined with each of at least one human hand image only once to obtainthe combination of at least one group of newly added chip images and thehuman hand image.

At S706, the time point when the number of at least one group of chipsis changed is marked as the trigger time point, and 6 image framescollected within 4 seconds before the trigger time point are obtained bymeans of the camera with aerial view and taken as at least one motiontrajectory image frame.

In the embodiments of the present disclosure, the second time period is4 seconds.

At S707, for recognized human hand images, the corresponding positioninformation of each human hand image in each of at least one motiontrajectory image frame is obtained.

In the embodiments of the present disclosure, for one human hand image,6 pieces of position information corresponding to the human hand imagemay be obtained from 6 motion trajectory image frames; and for therecognized human hand images, the respective corresponding positioninformation of the recognized human hand images in each of at least onemotion trajectory image frame is obtained.

At S708, according to a time sequence of collecting at least one motiontrajectory image frame, the position information corresponding to eachhuman hand image is sequentially connected, and the vector trajectoryobtained by means of the connection is taken as the actual motiontrajectory of each human hand image.

In the embodiments of the present disclosure, the time sequence of atleast one motion trajectory image frame is the time sequence ofcollecting at least one motion trajectory image frame.

At S709, in the combinations of at least one group of newly added chipimages and the human hand image, a path from the human hand image to thenewly added chip image in each combination is taken as the simulatedmotion trajectory from the human hand image to the newly added chipimage in each combination.

At S710, for the combination of each newly added chip image and thehuman hand image, the similarity between the simulated motion trajectoryin each combination and the actual motion trajectory of the human handimage is compared.

In the embodiments of the present disclosure, for each of thecombinations of at least one group of newly added chip images and thehuman hand image, the similarity between each actual motion trajectoryand each simulated motion trajectory may be compared by using the samemethod.

At S711, the combination having the highest degree of association withone actual motion trajectory is determined as the combination having thehighest degree of association.

At S712, the image frame to be matched corresponding to the combinationhaving the highest degree of association is taken as the associatedimage frame.

At S713, the human hand image in the combination having the highestdegree of association is determined as a target human hand image.

At S714, the image frame in the side view corresponding to theassociated image frame is obtained from the set of image frame in theside view, and the image frame in the side view corresponding to theassociated image frame includes the side image of the person (one playeror dealer) related with the newly added chip image.

The side image in the embodiments of the present disclosure also is theimage at the side view angle relate to the image at a top view angle.

In the embodiments of the present disclosure, the image frame in theside view corresponding to the associated image frame is the image framethat is collected with the side view camera and synchronized with theassociated image frame.

At S715, image fusion is performed on the associated image frame and theimage frame in the side view corresponding to the associated image frameto obtain the side image of a target human hand corresponding to thetarget human hand image in the image frame in the side viewcorresponding to the associated image frame.

In the embodiments of the present invention, the side view image of thetarget intervention part is the side image of the target human hand.

In the embodiments of the present disclosure, image fusion is performedon the associated image frame and the image frame in the side viewcorresponding to the associated image frame, and image informationassociation is performed on the human hand image of the associated imageframe at the bird's eye view and the human hand image of the side viewimage at the side view angle to obtain the corresponding image of thetarget human hand image in the image frame in the side viewcorresponding to the associated image frame at the bird's eye view,i.e., the side image of the target human hand.

At S716, the image of side body (i.e., the human body image at the sideview angle) of a target player associated with the side image of thetarget human hand is obtained on the basis of the image frame in theside view corresponding to the associated image frame.

In the embodiments of the present disclosure, the image frame in theside view corresponding to the associated image frame includes the sideimage of the human body of at least one person (the player or thedealer), and includes the side human hand image of at least one person,the side face image of at least one person, and the image of side bodyof at least one person, where the side human hand image, the side faceimage, and the image of side body of the same person are a wholeassociated with each other. On the basis of the image frame in the sideview corresponding to the associated image frame, the side image of theperson to which the side image of the target human hand belongs can befound, so that the image of side body of the target person associatedwith the side image of the target person is obtained.

At S717, on the basis of the image frame in the side view correspondingto the associated image frame, the face image of the target personassociated with the image of side body of the target person is obtained.

In the embodiments of the present invention, the side view image of thetarget intervention object is the side face image of the target person.

In the embodiments of the present disclosure, after the image of sidebody of the target person is obtained, the image of side body of thetarget person is associated to the face image of the target personcontinuously based on the association relationship between the image ofside body and the face image in the image frame in the side viewcorresponding to the associated image frame.

At S718, according to the face image of the target person and apre-stored corresponding relationship between images of interventionobjects and identity information of the intervention objects, theidentity information of the target person is determined.

In the embodiments of the present invention, the pre-storedcorresponding relationship between images of intervention objects andidentity information of the intervention objects is the pre-storedcorresponding relationship between the image of frontal face of eachperson and the identity information of the player.

It can be understood that by associating the target human hand image tothe side image of the target human hand at the side view angle, andaccording to the association relationship between the human hand, thehuman body and the human face in the image frame in the side viewcorresponding to the associated image frame, associating the side imageof the target human hand to the image of side body of the target person,the face image of the target person is further associated according tothe image of side body of the target player, and matching is performedaccording to the face image of the target person and the pre-storedimage of frontal face of at least one person, so that the target humanhand image obtained at the bird's eye view and associated with thechanged object is associated with the frontally collected image offrontal face of the target player, and the identity identification ofthe target person to which the changed object belongs is completed.

At S719, the side image of the newly added chip is obtained from theimage frame in the side view corresponding to the associated imageframe.

At S720, the object value of the newly added chip is obtained accordingto the side image of the newly added chip.

In the embodiments of the present disclosure, the newly added chipsinclude at least one chip having different colors or figures; each colorand figure of the chips represent different chip values; and the objectvalue of the newly added chip can be obtained according to the sideimage of the newly added chip.

At S721, the target person is associated with the object value to obtainthe information that the target person puts the chip of the objectvalue.

It can be understood that in the embodiments of the present disclosure,the situation that the new chips are stacked on the existing group ofchips in the detection area can be monitored by means of the side viewcamera, so that the situation of failing to effectively monitor thestacking of the chips by means of the camera with aerial view isavoided; meanwhile, the target human hand image is determined by meansof the motion trajectory from the human hand to the chip at the camerawith aerial view, so that it is avoided that a distortion occurs in themotion trajectory, which is obtained by means of the side view camera,from the human hand to the newly added chip, and that the associationbetween the newly added chip and the target human hand image isinaccurate; furthermore, the image of side body of the player isobtained by means of the side view camera, information fusion isperformed on the target human hand image at the bird's eye view and theimage of side body of the player at the side view angle, and theassociation between the target human hand image and the face image ofthe target person is established, so that by associating the face imageof the target person to the pre-stored image of frontal face of at leastone person, the target human hand image associated with the changedobject from the bird's eye view can be associated to the image offrontal face of the player, and finally, the accuracy of recognizing theidentity information of the person to which the changed object belongsis improved.

The embodiments of the present disclosure provide an image recognitionapparatus corresponding to the image recognition method; FIG. 7 is aschematic structural diagram I of the image recognition apparatusprovided by the embodiments of the present disclosure; and as shown inFIG. 7, the image recognition apparatus 4 includes the first determiningunit 400, the first obtaining unit 401, the second determining unit 402,the second obtaining unit 403, and the third determining unit 404.

The first determining unit 400 is configured to determine the changedobject according to the change of states of at least one group of targetobjects in the detection area.

The first determining unit 401 is configured to obtain the set of imageframe in the side view consisting of the image frames in the side viewthat are collected within the first preset time period before the statesof at least one group of target objects in the detection area arechanged, and at least one image frame to be matched, where at least oneimage frame to be matched is obtained by collecting an image of thedetection area by the image collection device located above thedetection area, and the image frames in the side view are obtained bycollecting images of the detection area by the image collection devicelocated on the side of the detection area.

The second determining unit 402 is configured to determine theassociated image frame from at least one image frame to be matched,where the associated image frame includes the intervention part havingthe highest degree of association with the changed object.

The second obtaining unit 403 is configured to obtain the image frame inthe side view corresponding to the associated image frame from the setof image frame in the side view, where the image frame in the side viewcorresponding to the associated image frame includes the interventionpart having the highest degree of association with the changed object,and at least one intervention object.

The third determining unit 404 is configured to determine, on the basisof the associated image frame and the image frame in the side viewcorresponding to the associated image frame, the target interventionobject having the highest degree of association with the changed objectfrom at least one intervention object.

In some embodiments, the image recognition apparatus 4 further includesa third obtaining unit, a fourth obtaining unit, and a first associatingunit.

The third obtaining unit is configured to obtain a side image of thechanged object from the image frame in the side view corresponding tothe associated image frame.

The fourth obtaining unit is configured to obtain an object value of thechanged object according to the side image of the changed object, wherethe object value is the sum of values of at least one sub changed objectconstituting the changed object, the side image of the changed objectincludes the side image of each of at least one sub changed object, andthe side image of each of at least one sub changed object representsdifferent values.

The first associating unit is configured to associate the targetintervention object and the object value.

In some embodiments, the second determining unit 402 includes arecognizing unit, a combining unit, a first determining sub unit, and asecond determining sub unit.

The recognizing unit is configured to recognize at least oneintervention part from each of at least one image frame to be matched,where the at least one intervention part is a part of the at least oneintervention object which appears in one image frame to be matched.

The combining unit is configured to combine the changed object and therecognized intervention part in the one-to-one manner to obtain at leastone combination of changed object and intervention part.

The first determining sub unit is configured to determine thecombination having the highest degree of association from the at leastone combination of changed object and intervention part, where theintervention part in the combination having the highest degree ofassociation is the target intervention part.

The second determining sub unit is further configured to take the imageframe to be matched in which the intervention part in the combinationhaving the highest degree of association is located as the associatedimage frame.

In some embodiments, the third determining unit 404 includes a fusingunit, a second associating unit, and a third determining sub unit.

The fusing unit is configured to perform image fusion on the associatedimage frame and the image frame in the side view corresponding to theassociated image frame to obtain a side view image of the targetintervention part, where the side view image of the target interventionpart is the image of the target intervention part in the image frame inthe side view corresponding to the associated image frame.

The second associating unit is configured to obtain, on the basis of theimage frame in the side view corresponding to the associated imageframe, the image in the side view of the target intervention objectassociated with the image in the side view of the target interventionpart, where the image frame in the side view corresponding to theassociated image frame includes the image in the side view of each of atleast one intervention object.

The third determining sub unit is configured to determine, according tothe image in the side view of the target intervention object and apre-stored corresponding relationship between images of interventionobjects and identity information of the intervention objects, identityinformation of the target intervention object.

In some embodiments, the first determining sub unit further includes:

a fifth obtaining unit, configured to obtain the distances between thechanged objects and the intervention parts in each of the at least onecombination of changed object and intervention part;

the fourth determining sub unit, further configured to determine theminimum distance from the distances; and

the fifth determining sub unit, further configured to take thecombination of the changed object and the intervention partcorresponding to the minimum distance as the combination having thehighest degree of association.

In some embodiments, the fourth determining sub unit is configured to:

perform inverse ratio calculation on the distance to obtain thecalculation results of each of the distances; and

determine the maximum calculation result from the calculation results,and determine the distance corresponding to the maximum calculationresult as the minimum distance.

In some embodiments, the first determining sub unit further includes asixth obtaining unit, a seventh obtaining unit, a comparing unit, and asixth determining sub unit, where

the sixth obtaining unit is configured to obtain the actual motiontrajectory of each of the recognized intervention parts;

the seventh obtaining unit is configured to obtain, in the at least onecombination of changed object and intervention part, the simulatedmotion trajectory from the intervention part to the changed object ineach combination, where the simulated motion trajectory from theintervention part to the changed object in one combination is thetrajectory connecting the intervention part and the changed object inthe combination;

the comparing unit is configured to compare the similarity between eachactual motion trajectory and each simulated motion trajectory; and

the sixth determining sub unit is configured to determine thecombination of the changed object and the intervention partcorresponding to the simulated motion trajectory having the highestsimilarity with the actual motion trajectory as the combination havingthe highest degree of association.

In some embodiments, the sixth obtaining unit includes an eighthobtaining unit, a ninth obtaining unit, a tenth obtaining unit, and aconnecting unit, where

the eighth obtaining unit is configured to obtain the trigger time pointwhen the state of the target object is changed;

the ninth obtaining unit is configured to obtain at least one motiontrajectory image frame that is collected within the second preset timeperiod before the trigger time point, where the at least one motiontrajectory image frame is obtained by the image collection devicelocated above the detection area;

the tenth obtaining unit is configured to obtain the correspondingposition information of each recognized intervention part in at leastone motion trajectory image frame; and

the connecting unit is configured to perform, according to a timesequence of at least one motion trajectory image frame, vectorizationconnection on the position information to obtain the actual motiontrajectory of each recognized intervention part.

In some embodiments, the image recognition apparatus 4 further includesa first monitoring unit, where

the first monitoring unit is configured to determine, when it ismonitored, by means of the image collection device located above thedetection area, that one new group of target objects appear in thedetection area or that any of the at least one group of previous targetobjects in the detection area disappear, that the state of at least onegroup of target objects in the detection area is changed.

In some embodiments, at least one group of target objects are formed bystacking at least one sub object, and the image recognition apparatus 4further includes a second monitoring unit, where

the second monitoring unit is configured to determine, when it ismonitored, by means of the image collection device located on the sideof the detection area, that the number or the appearance of sub objectsincluded in any of the at least one group of target objects is changed,that the states of the at least one group of target objects in thedetection area are changed.

It should be noted that in actual application, the first determiningunit 400, the first obtaining unit 401, the second determining unit 402,the second obtaining unit 403, and the third determining unit 404 may berealized by means of the processor 54 located at the image recognitionapparatus 2, and specifically are realized by a Central Processing Unit(CPU), a Microprocessor Unit (MPU), a Digital Signal Processing (DSP),or a Field Programmable Gate Array (FPGA).

The embodiments of the present disclosure provide an image recognitionapparatus corresponding to the image recognition method; FIG. 8 is aschematic structural diagram II of the image recognition apparatusprovided by the embodiments of the present disclosure; as shown in FIG.8, the image recognition apparatus 5 includes the processor 54, thememory 55, and the communication bus 56; the memory 55 communicates withthe processor 54 by means of the communication bus 56; the memory 55stores one or more programs that may be executed by the processor 54;and if one or more programs are executed, the processor 54 executes theaforementioned any one image recognition method.

It can be understood that in the embodiments of the present disclosure,the image recognition apparatus can obtain the information of the targetintervention part having higher accuracy from the bird's eye view, andby means of image fusion, combines the information of the targetintervention part at the bird's eye view with the complete sideinformation of the intervention object at the side view angle, so thatthe present disclosure realizes the association from the information ofthe target intervention part at the bird's eye view to the informationof the target intervention object at the side view angle, furtherassociating the pre-stored image information of the target interventionobject, and improving the accuracy of recognizing the targetintervention object.

The embodiments of the present disclosure provide a computer-readablestorage medium, where the computer-readable storage medium stores one ormore programs; one or more programs may be executed by one or moreprocessors 54; and the image recognition method, as shown in theembodiments of the present disclosure, is realized when the program isexecuted by the processor 54.

A person skilled in the art should understand that the embodiments ofthe present disclosure may provide a method, a system or a computerprogram product. Therefore, the present disclosure may use the forms ofhardware embodiments, software embodiments, or the embodiments ofcombining software and hardware aspects. Moreover, the presentdisclosure may use the form of the computer program product implementedover one or more computer usable storage mediums (including but notlimited to a disk memory and an optical memory, etc.) that include acomputer usable program code.

The present disclosure is described with reference to flowcharts and/orblock diagrams of methods, devices (systems), and computer programproducts of the embodiments of the present disclosure. It should beunderstood that a computer program instruction is configured toimplement each flow and/or block in the flowcharts and/or blockdiagrams, and the combination of flows/blocks in the flowcharts and/orblock diagrams. These computer program instructions may be provided to auniversal computer, a special computer, an embedded processor orprocessors of other programmable data processing devices to generate amachine such that an apparatus for implementing the functions specifiedin one or more flow in the flowcharts and/or one or more blocks in theblock diagrams is generated through the instructions executed by thecomputer or the processor of other programmable data processing devices.

These computer program instructions may also be stored in a computerreadable memory that can direct the computer or other programmable dataprocessing devices to work in a particular manner such that theinstruction stored in the computer readable memory generates a productincluding an instruction apparatus, which implements the functionsspecified in one or more flows in the flowchart and/or one or moreblocks in the block diagram.

These computer program instructions may also be loaded in a computer orother programmable data processing devices such that a series ofoperation operations are executed on the computer or other programmabledata processing devices to generate computer implemented processing, andthus the instruction executed on the computer or other programmable dataprocessing devices provides the operations for implementing thefunctions specified in one or more flows in the flowchart and/or one ormore blocks in the block diagram.

The foregoing descriptions are merely preferred embodiments of thepresent disclosure, but are not intended to limit the scope ofprotection of the present disclosure.

1. An image recognition method, comprising: determining a changed objectaccording to a change of states of at least one group of target objectsin a detection area; obtaining at least one image frame to be matchedand a set of image frames in a side view consisting of image frames inthe side view that are collected within a first preset time periodbefore the states of the at least one group of target objects in thedetection area are changed, wherein the at least one image frame to bematched is obtained by collecting at least one image of the detectionarea by an image collection device located above the detection area, andthe image frames in the side view are obtained by collecting images ofthe detection area by an image collection device located on a side ofthe detection area; determining an associated image frame from the atleast one image frame to be matched, wherein the associated image framecomprises an intervention part having a highest degree of associationwith the changed object; obtaining an image frame in the side viewcorresponding to the associated image frame from the set of image framesin the side view, wherein the image frame in the side view correspondingto the associated image frame comprises the intervention part having thehighest degree of association with the changed object, and at least oneintervention object; and determining, according to the associated imageframe and the image frame in the side view corresponding to theassociated image frame, a target intervention object having the highestdegree of association with the changed object from the at least oneintervention object.
 2. The method according to claim 1, wherein afterthe determining, according to the associated image frame and the imageframe in the side view corresponding to the associated image frame, thetarget intervention object having the highest degree of association withthe changed object from the at least one intervention object, the methodfurther comprises: obtaining a side image of the changed object from theimage frame in the side view corresponding to the associated imageframe; obtaining an object value of the changed object according to theside image of the changed object, wherein the object value is a sum ofvalues of at least one sub changed object constituting the changedobject, the side image of the changed object comprises a side image ofeach of the at least one sub changed object, and the side image of eachof the at least one sub changed object represents different values; andassociating the target intervention object with the object value.
 3. Themethod according to claim 1, wherein the determining the associatedimage frame from the at least one image frame to be matched comprises:recognizing at least one intervention part from each of the at least oneimage frame to be matched, wherein the at least one intervention part isa part of the at least one intervention object which appears in oneimage frame to be matched; combining the changed object and therecognized at least one intervention part in a one-to-one manner toobtain at least one combination of changed object and intervention part;determining a combination having a highest degree of association fromthe at least one combination of changed object and intervention part,wherein an intervention part in the combination having the highestdegree of association is a target intervention part; and taking an imageframe to be matched in which the intervention part in the combinationhaving the highest degree of association is located as the associatedimage frame.
 4. The method according to claim 3, wherein thedetermining, according to the associated image frame and the image framein the side view corresponding to the associated image frame, the targetintervention object having the highest degree of association with thechanged object from the at least one intervention object comprises:performing image fusion on the associated image frame and the imageframe in the side view corresponding to the associated image frame toobtain an image in the side view of the target intervention part,wherein the image in the side view of the target intervention part is animage of the target intervention part in the image frame in the sideview corresponding to the associated image frame; obtaining, accordingto the image frame in the side view corresponding to the associatedimage frame, an image in the side view of the target intervention objectassociated with the image in the side view of the target interventionpart, wherein the image frame in the side view corresponding to theassociated image frame comprises an image in the side view of each ofthe at least one intervention object; and determining, according to theimage in the side view of the target intervention object and apre-stored corresponding relationship between images of interventionobjects and identity information of the intervention objects, identityinformation of the target intervention object.
 5. The method accordingto claim 3, wherein the determining the combination having the highestdegree of association from the at least one combination of changedobject and intervention part comprises: obtaining distances between thechanged object and the intervention part in each of the at least onecombination of changed object and intervention part; determining aminimum distance from the distances; and taking the combination ofchanged object and intervention part corresponding to the minimumdistance as the combination having the highest degree of association. 6.The method according to claim 3, wherein the determining the combinationhaving the highest degree of association from the at least onecombination of changed object and intervention part comprises: obtainingan actual motion trajectory of each of the recognized at least oneintervention part; obtaining, in the at least one combination of changedobject and intervention part, a simulated motion trajectory from theintervention part to the changed object in each of the at least onecombination of changed object and intervention part, wherein thesimulated motion trajectory from the intervention part to the changedobject in one combination is a trajectory connecting the interventionpart and the changed object in the combination; comparing a similaritybetween each actual motion trajectory and each simulated motiontrajectory; and determining the combination of the changed object andthe intervention part corresponding to the simulated motion trajectoryhaving the highest similarity with the actual motion trajectory as thecombination having the highest degree of association.
 7. The methodaccording to claim 6, wherein the obtaining the actual motion trajectoryof each of the recognized at least one intervention part comprises:obtaining a trigger time point responsive to a state of the targetobject being changed; obtaining at least one motion trajectory imageframe that is collected within a second preset time period before thetrigger time point, wherein the at least one motion trajectory imageframe is obtained by the image collection device located above thedetection area; obtaining corresponding position information of each ofthe recognized at least one intervention part in the at least one motiontrajectory image frame; and performing, according to a time sequence ofthe at least one motion trajectory image frame, vectorization connectionon the position information to obtain the actual motion trajectory ofeach of the recognized at least one intervention part.
 8. The methodaccording to claim 1, wherein the change of states of the at least onegroup of target objects in the detection area comprises: responsive tomonitoring, by means of the image collection device located above thedetection area, that a new group of target objects appear in thedetection area or that any of the at least one group of the targetobjects in the detection area disappear, determining that the states ofthe at least one group of target objects in the detection area arechanged.
 9. The method according to claim 1, wherein the at least onegroup of target objects are formed by stacking at least one sub object,and the change of states of the at least one group of target objects inthe detection area comprises: responsive to monitoring, by means of theimage collection device located on the side of the detection area, thata number or an appearance of sub objects comprised in any of the atleast one group of target objects is changed, determining that thestates of the at least one group of target objects in the detection areaare changed.
 10. An image recognition apparatus, comprising a processor,a memory, and a communication bus, wherein the memory communicates withthe processor by means of the communication bus; the memory stores oneor more programs that are executable by the processor; and when the oneor more programs are executed, the processor is configured to: determinea changed object according to a change of states of at least one groupof target objects in a detection area; obtain at least one image frameto be matched and a set of image frames in a side view consisting ofimage frames in the side view that are collected within a first presettime period before the states of the at least one group of targetobjects in the detection area are changed, wherein the at least oneimage frame to be matched is obtained by collecting at least one imageof the detection area by an image collection device located above thedetection area, and the image frames in the side view are obtained bycollecting images of the detection area by an image collection devicelocated on a side of the detection area; determine an associated imageframe from the at least one image frame to be matched, wherein theassociated image frame comprises an intervention part having a highestdegree of association with the changed object; obtain an image frame theside view corresponding to the associated image frame from the set ofimage frames in the side view, wherein the image frame in the side viewcorresponding to the associated image frame comprises the interventionpart having the highest degree of association with the changed object,and at least one intervention object; and determine, according to theassociated image frame and the image frame in the side viewcorresponding to the associated image frame, a target interventionobject having the highest degree of association with the changed objectfrom the at least one intervention object.
 11. The image recognitionapparatus according to claim 10, wherein the processor is furtherconfigured to: obtain a side image of the changed object from the imageframe in the side view corresponding to the associated image frame;obtain an object value of the changed object according to the side imageof the changed object, wherein the object value is a sum of values of atleast one sub changed object constituting the changed object, the sideimage of the changed object comprises a side image of each of the atleast one sub changed object, and the side image of each of the at leastone sub changed object represents different values; and associate thetarget intervention object and the object value.
 12. The imagerecognition apparatus according to claim 10, wherein the processor isspecifically configured to: recognize at least one intervention partfrom each of at least one image frame to be matched, wherein the atleast one intervention part is a part of the at least one interventionobject which appears in one image frame to be matched; combine thechanged object and the recognized at least one intervention part in aone-to-one manner to obtain at least one combination of changed objectand intervention part; determine a combination having a highest degreeof association from the at least one combination of changed object andintervention part, wherein an intervention part in the combinationhaving the highest degree of association is a target intervention part;and take an image frame to be matched in which the intervention part inthe combination having the highest degree of association is located asthe associated image frame.
 13. The image recognition apparatusaccording to claim 12, wherein the processor is specifically configuredto: perform image fusion on the associated image frame and the imageframe in the side view corresponding to the associated image frame toobtain an image in the side view of the target intervention part,wherein the image in the side view of the target intervention part is animage of the target intervention part in the image frame in the sideview corresponding to the associated image frame; obtain, according tothe image frame in the side view corresponding to the associated imageframe, an image in the side view of the target intervention objectassociated with the image in the side view of the target interventionpart, wherein the image frame in the side view corresponding to theassociated image frame comprises an image in the side view of each ofthe at least one intervention object; and determine, according to theimage in the side view of the target intervention object and apre-stored corresponding relationship between images of interventionobjects and identity information of the intervention objects, identityinformation of the target intervention object.
 14. The image recognitionapparatus according to claim 12, wherein the processor is specificallyconfigured to: obtain distances between the changed objects and theintervention parts in each of the at least one combination of changedobject and intervention part; determine a minimum distance from thedistances; and take the combination of changed object and interventionpart corresponding to the minimum distance as the combination having thehighest degree of association.
 15. The image recognition apparatusaccording to claim 12, wherein the processor is specifically configuredto: obtain an actual motion trajectory of each of the recognized atleast one intervention parts; obtain, in the at least one combination ofchanged object and intervention part, a simulated motion trajectory fromthe intervention part to the changed object in each of the at least onecombination of changed object and intervention part, wherein thesimulated motion trajectory from the intervention part to the changedobject in one combination is a trajectory connecting the interventionpart and the changed object in the combination; compare a similaritybetween each actual motion trajectory and each simulated motiontrajectory; and determine the combination of the changed object and theintervention part corresponding to the simulated motion trajectoryhaving the highest similarity with the actual motion trajectory as thecombination having the highest degree of association.
 16. The imagerecognition apparatus according to claim 15, wherein the processor isspecifically configured to: obtain a trigger time point responsive to astate of the target object being changed; obtain at least one motiontrajectory image frame that is collected within a second preset timeperiod before the trigger time point, wherein the at least one motiontrajectory image frame is obtained by the image collection devicelocated above the detection area; obtain corresponding positioninformation of each of the recognized at least one intervention part inthe at least one motion trajectory image frame; and perform, accordingto a time sequence of the at least one motion trajectory image frame,vectorization connection on the position information to obtain theactual motion trajectory of each of the recognized at least oneintervention part.
 17. The image recognition apparatus according toclaim 10, wherein the processor is further configured to: determine,responsive to monitoring, by means of the image collection devicelocated above the detection area, that a new group of target objectsappear in the detection area or that any of the at least one group ofthe target objects in the detection area disappear, that the states ofthe at least one group of target objects in the detection area arechanged.
 18. The image recognition apparatus according to claim 10,wherein the at least one group of target objects are formed by stackingat least one sub object, and the processor is further configured to:determine, responsive to monitoring, by means of the image collectiondevice located on the side of the detection area, that a number or anappearance of sub objects comprised in any of the at least one group oftarget objects is changed, that the states of the at least one group oftarget objects in the detection area are changed.
 19. A non-transitorycomputer-readable storage medium, wherein the non-transitorycomputer-readable storage medium stores one or more programs, and theone or more programs are executable by one or more processors toimplement the following operations: determining a changed objectaccording to a change of states of at least one group of target objectsin a detection area; obtaining at least one image frame to be matchedand a set of image frames in a side view consisting of image frames inthe side view that are collected within a first preset time periodbefore the states of the at least one group of target objects in thedetection area are changed, wherein the at least one image frame to bematched is obtained by collecting at least one image of the detectionarea by an image collection device located above the detection area, andthe image frames in the side view are obtained by collecting images ofthe detection area by an image collection device located on a side ofthe detection area; determining an associated image frame from the atleast one image frame to be matched, wherein the associated image framecomprises an intervention part having a highest degree of associationwith the changed object; obtaining an image frame in the side viewcorresponding to the associated image frame from the set of image framesin the side view, wherein the image frame in the side view correspondingto the associated image frame comprises the intervention part having thehighest degree of association with the changed object, and at least oneintervention object; and determining, according to the associated imageframe and the image frame in the side view corresponding to theassociated image frame, a target intervention object having the highestdegree of association with the changed object from the at least oneintervention object.
 20. The non-transitory computer-readable storagemedium according to claim 19, wherein after the determining, accordingto the associated image frame and the image frame in the side viewcorresponding to the associated image frame, the target interventionobject having the highest degree of association with the changed objectfrom the at least one intervention object, the operations furthercomprises: obtaining a side image of the changed object from the imageframe in the side view corresponding to the associated image frame;obtaining an object value of the changed object according to the sideimage of the changed object, wherein the object value is a sum of valuesof at least one sub changed object constituting the changed object, theside image of the changed object comprises a side image of each of theat least one sub changed object, and the side image of each of the atleast one sub changed object represents different values; andassociating the target intervention object with the object value.